Feature Weighting for Lazy Learning Algorithms

Learning algorithms diier in the degree to which they process their inputs prior to their use in performance tasks. Many algorithms eagerly compile input samples and use only the compilations to make decisions. Others are lazy: they perform less precompilation and use the input samples to guide decision making. The performance of many lazy learners signiicantly degrades when samples are deened by features containing little or misleading information. Distinguishing feature relevance is a critical issue for these algorithms, and many solutions have been developed that assign weights to features. This chapter introduces a cat-egorization framework for feature weighting approaches used in lazy similarity learners and brieey surveys some examples in each category. 1.1 INTRODUCTION Lazy learning algorithms are machine learning algorithms (Mitchell, 1997) that are welcome members of procrastinators anonymous. Purely lazy learners typically display the following characteristics (Aha, 1997): 1. Defer: They delay the processing of their inputs until they receive requests for information; they simply store their inputs for future use. 2. Demand-Driven: They reply to information queries by combining information from their stored (e.g., training) samples. 3. Discard: They delete the constructed query and any intermediate results. In contrast, eager algorithms greedily replace their inputs with an abstraction (e.g., a rule set, decision tree, or neural network) and use it to process queries.